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ABSTRACT 

1 

A  validation  was  conducted  on  the  air  defense  artillery  (ADA) 
section  of  the  Joint  Exercise  Support  System  (JESS)  model  version 
1.1. \  As  a  result  of  this  study,  we  recommend  that  JESS  1.1  ADA 
nojt  be  accepted  as  valid. 

The  data  base  contains  weapon  characteristic  values  that  are 
producing  invalid  results.  The  Pk  values  for  the  ADA  systems 
need  to  be  degraded  and  the  rounds  per  burst  for  the  guns  need  to 
be  decreased. 

Errors  in  the  JESS  1.1  code  are  also  producing  invalid  results. 

1.  The  range  degradation  factors  for  ADA  weapons  are  not 
being  correctly  applied.  Consequently,  long-range  ADA  weapons 
are  allowed  to  fire  at  aircraft  flying  nap  of  earth  (NOE) . 

2.  Helicopters  flying  at  NOE  are  more  vulnerable  than 
helicopters  flying  at  200  feet  above  ground  level. 

3.  The  definition  of  altitude  band  boundaries  in  the  data 
base  is  not  consistent  with  the  code  logic  used  to  determine  the 
altitude  band  and  used  to  select  the  pk  which  is  dependent  upon 
altitude.  This  results  in  erroneous  data  being  supplied  to  the 
data  base.  Providing  some  consistency  in  the  definition  of  these 
altitude  bands  would  aid  the  people  who  supply  Pk  data  based  upon 
aircraft  altitude. 

4.  The  code  allows  aircraft  to  fly  below  ground  level.  This 
results  in  negative  training  and  unintended  protection  to 
aircraft  since  ADA  weapons  will  not  fire  at  aircraft  flying  below 
a  minimum  altitude. 

The  results  from  the  JESS  1.1  ADA  tests  were  compared  to  a 
benchmark  set  of  ADA  results  obtained  from  the  Vector-in- 
Commander  (VIC)  model.  A  previous  report  entitled  Technical 
Memorandum  TRAC-F-TM-1388  JESS  1.0  Air  Defense  Artillery  (ADA) 
Validation,  published  in  October  1988,  describes  how  this 
benchmark  was  developed. 

Sensitivity  tests  were  conducted  on  the  JESS  1.1  ADA  module  to 
determine  the  effect  of  changing  mission  type  and  size,  altitude, 
number  of  weapons,  aircraft  type,  range  and  Pk  of  weapon,  and 
number  of  rounds  per  burst  on  attrition. 
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1.  Purpose.  This  report  presents  the  results  of  the  December 
1988  validation  tests  performed  on  the  air  defense  artillery  (ADA) 
play  in  the  Joint  Exercise  Support  System  (JESS)  1.1.  The  purpose 
of  this  testing  was  to  compare  the  results  from  JESS  1.1  to 
results  in  a  previously  published  run.  Technical  Memorandum 
TRAC-P-TM-1388,  October  1988,  entitled  JESS  1.0  Air  Defense 
Artillery  (ADA)  Validation.  This  classified  memorandum  contains 
results  from  JESS  1.0  air  defense  play  and  the  Vector- I n-Commander 
(VIC)  6.3  excursion  run  which  was  used  as  a  benchmark  to  judge  the 
validation  of  the  JESS  1.0  results.  These  VIC  results  were  also 
used  to  judge  the  results  from  our  December  tests  on  JESS  1.1. 

The  results  from  this  analysis  will  be  presented  to  the  JESS 
configuration  control  board  and  other  users. 

2.  Problem.  Several  changes  have  been  made  to  the  JESS  1.0 
version  of  the  air  defense  module  in  JESS  1.1.  We  felt  it  was 
necessary  to  repeat  the  validation  testing  that  had  been  done  on 
JESS  1.0  to  determine  the  significance  of  these  changes. 

a.  In  JESS  1.0,  helicopters  were  modeled  the  same  as 
fixed-wing  aircraft.  In  JESS  1.1,  helicopters  are  modeled 
separately  from  fixed-wing  aircraft.  Orders  for  helicopter 
missions  are  processed  under  the  army  aviation  menu  which  is 
different  from  a  fixed-wing  order  menu,  and  ADA  damage  reports 
for  helicopters  are  processed  separately  from  fixed-wing  damage 
reports. 

b.  A  new  table  of  data  was  added  to  JESS  1.1  to  allow  for 
range  degradation  of  ADA  weapons,  depending  upon  the  aircraft's 
altitude.  This  table  of  degradation  factors  reduces  the  lethal 
range  of  some  ADA  weapons  to  zero  if  the  aircraft  is  flying  nap 
of  earth  (NOE)  altitude. 

c.  Lastly,  we  learned  that  the  Jet  Propulsion  Laboratory 
(JPL)  had  rewritten  the  code  that  calculates  the  number  of  rounds 
fired  per  salvo.  In  JESS  1.0,  the  number  of  rounds  fired  per 
salvo  depended  upon  the  number  of  weapon  systems  in  the  ADA  unit. 
If  a  unit  had  six  Hawks,  six  shots  would  be  fired  for  each  salvo. 
We  were  told  this  methodology  had  been  changed.  For  missiles,  the 
number  of  shots  fired  is  dependent  upon  a  value  assigned  in  the 
data  base  to  that  weapon  system.  For  guns,  the  old  methodology 
still  applies  (i.e.,  one  burst  times  the  number  of  guns  in  the 
unit  for  each  salvo  fired) . 

3.  Objectives.  Our  objectives  were  threefold. 

a.  To  determine  what  impact  the  above  changes  had  upon  ADA 
attrition  results  compared  to  ADA  play  in  JESS  1.0. 

b.  To  compare  the  results  from  JESS  1.1  to  the  VIC  benchmark 
results  (previously  published  in  Technical  Memorandum  TRAC-F-TM 
1388,  October  1988)  . 


c.  To  conduct  sensitivity  tests  on  both  gaaer  input 
variables  and  data  base  variables  such  as  weapon  range  and  rounds 
per  burst. 

4.  Limitations.  The  aircraft  and  ADA  weapons  tested  were 
limited  to  those  which  are  common  to  the  VIC  6.3  excursion 
benchmark  and  JESS  1.1/  with  the  exception  of  the  Vulcan  which  is 
not  played  in  VIC.  This  analysis  did  not  include  synergistic 
effects  on  ADA  units  by  other  combat  units.  The  units  in  the 
analysis  were  ccxnposed  of  only  one  type  of  ADA  weapon  system. 

Only  close  air  support  (CAS)  and  battlefield  air  interdiction 
(BAD  missions  were  tested  in  this  analysis.  Our  tests  were 
confined  to  the  two  lower  altitude  bands  (0  to  490  and  490  to 
4,900  feet) .  No  air  corridor  protection  to  missions  was  provided 
in  our  tests.  The  tests  were  structured  so  no  attrition  of  ADA 
units  occurred  from  enemy  fire  during  our  tests. 

5.  Assumptions . 

a.  The  ADA  unit  configurations  in  VIC  excursion  6.3  are 
correct  and  acceptable  for  our  analysis. 

b.  The  VIC  ADA  results  from  the  Europe  6.3  base  case 
presented  in  the  JESS  1.0  ADA  Validation  Report  (TRAC-F-IM-1388 ) 
are  acceptable  for  use  as  a  benchmark  in  this  analysis. 

6.  Constraints.  Analysis  of  the  ADA  module  of  JESS  1.1  was 
subject  to  the  same  general  constraints  as  the  analysis  of  the 
other  modules.  Source  code  was  not  available.  We  used  the 
Software  Design  and  Documentation  Language  (SDDL)  code  for  this 
analysis . 

7.  Methodology.  For  this  analysis  a  data  base  was  constructed 
containing  ADA  units  composed  of  only  one  type  of  ADA  weapon 
system  and  the  radars  associated  with  that  system.  The  number  of 
weapon  systems  in  the  unit  was  the  same  as  the  number  in  the 
benchmark  VIC  run.  The  ADA  units  were  located  far  enough  from 
each  other  to  prevent  more  than  one  ADA  unit  firing  at  a  mission. 
Each  air  mission  or  army  aviation  mission  was  assigned  an  ADA 
unit  as  a  target  to  make  sure  the  mission  flew  dead  center  over 
the  ADA  unit.  The  air  missions  were  given  minimal  munitions  to 
fire  to  avoid  degradation  to  the  target  ADA  unit.  Each  air 
mission  or  army  aviation  mission  was  replicated  10  times  to 
obtain  an  average  result.  Some  tests  were  repeated  an  additional 
10  times  to  test  the  effect  of  sample  size  on  the  results. 

a.  Related  studies.  JESS  1.0  Air  Defense  Artillery  (ADA) 
Validation  {TRAC-F-TM-13  88)  ,  CX:tober  1988. 

b.  Essential  elements  of  analysis  (EEA)  and  measures  of 
effectiveness  (MOE) . 

(1)  EEA.  The  following  EEA  were  used  to  determine  what 
results  were  needed  in  the  analysis: 
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(a)  How  does  attrition  of  aircraft  in  JESS  1.1  compare 
with  attrition  of  aircraft  in  VIC  and  in  JESS  1.0. 

(b)  In  an  engagement,  over  what  range  does  attrition  to 
aircraft  occur? 

(c)  How  many  shots  are  fired  by  the  ADA  unit  in  an 
engagement? 

(d)  How  successful  are  ADA  units  in  detecting  air 
missions? 

(2)  The  MOE  used  in  the  analysis  were  as  follows: 

(a)  What  is  the  average  number  of  aircraft  killed  in  an 
engagement  with  each  ADA  unit  (expressed  as  percent  of  total 
aircraft  flown)? 

(b)  What  is  the  average  number  of  aircraft  hit  in  each 
ADA  engagement? 

(c)  What  is  the  minimum  and  maximum  number  of  aircraft 
hit  in  an  engagement  with  each  ADA  unit? 

(d)  What  is  the  average  number  of  shots  fired  per 
engagement? 

(e)  What  is  the  average  percent  of  air  missions  detected 
by  the  radar  units? 

(f)  What  is  the  minimum  and  maximum  range  of  detection 
for  each  set  of  10  samples? 

(g)  What  is  the  minimum  and  maximum  range  the  ADA  weapon 
fires  for  each  set  of  ten  samples? 

c.  Method  of  analysis.  The  following  aircraft  were  used  in 
the  tests:  A-10,  F-16,  AH-64,  Frogfoot,  and  Hind.  The  following 
ADA  weapons  were  used  in  the  tests:  Patriot,  Hawk,  Chaparral, 
Stinger,  Vulcan,  SA-11,  SA-13,  SA-14,  and  ZSU  23-4.  Air  missions 
contained  two  aircraft  per  mission,  except  in  sensitivity  tests 
designed  to  test  the  effect  of  mission  size  on  attrition  from 
ADA.  Army  aviation  missions  contained  four  aircraft  per  mission 
except  in  sensitivity  tests  designed  to  test  the  effect  of 
mission  size  on  attrition  from  ADA.  The  base  case  in  this 
analysis  consists  of  15  tests.  Nine  of  these  tested  fixed-wing 
aircraft  against  nine  different  ADA  weapon  systems;  six  tested 
helicopters  against  six  different  ADA  weapon  systems. 

Sensitivity  tests  were  run  to  determine  the  effect  of  changing 
the  mission  parameters  or  the  ADA  weapon  characteristics.  The 
sensitivity  runs  included  the  following; 

(1)  Effect  of  mission  type  (BAI  or  CAS) . 
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(2)  Effect  of 
(3>  Effect  of 

(4)  Effect  of 

(5)  Effect  of 

(6)  Effect  of 

(7)  Effect  of 
an  ADA  weapon. 

(8)  Effect  of 
the  ADA  weapon. 


mission  size, 
aircraft  altitude. 

number  of  weapon  systems  in  the  ADA  unit, 
aircraft  type  (F-16  versus  A-10) . 
changing  the  range  of  an  ADA  weapon, 
changing  the  number  of  rounds  per  burst  of 

changing  the  probability  of  kill  (Pk)  of 


8.  Results.  First,  to  establish  that  a  sample  size  of  10  was 
sufficient  for  determining  the  average  number  of  aircraft  killed 
and  hit,  we  ran  three  tests  with  a  sample  size  of  20  missions  and 
compared  the  results  to  a  10-sample  test  run.  The  results  are  in 
table  1.  The  results  show  at  most  a  10  percent  difference 
between  a  sample  size  of  10  and  20  in  the  number  of  hits  for  a 
Stinger.  We  accepted  this  range  in  results  and  opted  to  use  the 
smaller  sample  size  of  10  for  our  tests.  This  allowed  us  to 
increase  the  number  of  sensitivity  test  runs,  given  our  limited 
resources. 


Table  1.  Effect  of  sample  size  on  attrition  results 

Sample 


ADA  vs.  mission _ size _ %  hit 

SA-14  vs.  AH-64  10  23 

20  21 

Stinger  vs.  Frogfoot  10  25 

20  28 

Stinger  vs.  Hind  10  58 

20  56 


a.  Base-case  attrition  results.  Tables  2  and  3  list  the 
results  from  running  the  test  matrix  with  respect  to  average 
percent  of  aircraft  killed  and  hit  in  the  army  aviation  and 
fixed-wing  aircraft  tests.  The  tables  list  the  weapon,  the 
mission  flown  against  the  ADA  weapon  type,  the  average  percent  of 
aircraft  killed,  the  average  percent  of  aircraft  hit,  the  minimum 
number  hit  in  any  one  mission,  the  maximum  number  hit  in  any  one 
mission,  and  the  average  number  of  rounds  fired  at  the  mission  by 
the  ADA  unit.  These  averages  were  found  by  dividing  the  total 
number  of  aircraft  hit  or  killed  over  10  missions  by  the  total 
number  of  aircraft  flown  in  10  missions.  The  tables  list  the 
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killers  of  aircraft  in  descending  order  of  lethality  sorted  by 
side  (Red,  ^^ue)  and  similar  test  parameters  (NOE,  200  feet, 
eight  helicopters,  etc.) 

(1)  In  two  ranking  cases,  the  number  of  aircraft  hit  was 
used  to  break  a  tie  between  weapon  systems  with  approximately 
equal  kills.  In  table  2,  the  SA-11  and  SA-13  had  equal  kills  but 
varied  significantly  in  the  number  hit.  In  table  3,  the 
Chaparral  and  Vulcan  killed  approximately  the  same  number  of 
aircraft  but  varied  significantly  in  the  number  hit. 

(2)  The  minimum  and  maximum  number  hit  and  the  average 
number  of  shots  fired  per  engagement  were  used  to  help  determine 
if  the  differences  in  number  hit  or  killed  were  significant.  An 
engagement  is  defined  as  the  total  number  of  salvos  fired  at  the 
mission  while  it  is  within  range  of  the  ADA  unit. 

(3)  When  an  aircraft  is  hit,  a  random  draw  is  made  to 
determine  if  the  hit  aircraft  has  been  killed  or  just  damaged. 
Each  aircraft  type  has  a  probability  of  damage  value  assigned  to 
it  in  the  data  base.  Damaged  aircraft  return  to  the  base  for 
repair  and  can  re-enter  the  game  after  repairs.  Repair  time  is 
determined  by  a  random  draw  and  varies  from  a  few  hours  to 
infinity  (these  aircraft  are  not  repairable).  Damaged  aircraft 
immediately  drop  out  of  the  mission  and  return  to  base.  They  do 
not  attempt  to  deliver  ordnance.  From  that  particular  mission's 
viewpoint,  a  damaged  aircraft  does  not  contribute  any  more  to  the 
mission  than  a  killed  aircraft.  A  damaged  aircraft  can  be  shot 
again  by  the  ADA  unit  if  it  is  within  range  of  the  weapon  but 
only  if  there  are  no  other  undamaged  aircraft  to  shoot  at.  Once 
a  damaged  aircraft  is  hit  again,  it  becomes  a  killed  aircraft. 

It  is  possible  to  hit  a  mission  with  four  aircraft  more  than  four 
times  if  the  hits  result  in  damage  and  not  kills. 

(a)  Army  aviation.  The  results  from  table  2  show  that 
comparable  Red  and  Blue  ADA  weapons  are  not  comparably  hitting  or 
killing  helicopter  targets.  The  Stinger  is  comparable  to  the 
SA-14,  yet  the  results  show  the  Stinger  is  twice  as  lethal  as  the 
SA-14  in  killing  helicopters.  The  ZSU-23-4  and  the  Vulcan  are 
comparable  weapons.  They  hit  about  the  same  number  of 
helicopters  flying  NOE.  The  Chaparral,  SA-11,  and  SA-13  are  not 
supposed  to  be  able  to  shoot  at  helicopters  flying  NOE  if  the 
game  is  correctly  using  the  range  degrader  bands  in  the  data 
base.  The  results  in  table  2  show  these  systems  are  major 
killers  of  helicopters  flying  NOE. 
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Table  2.  Helicopter  ADA  results  from  JESS  1.1 

Ave.  %  Ave.  %  Min  no.  Max  no.  Ave  no. 
Weapon  vs.  tgt  killed  hit  hit  hit  rds. 

Red  ADA  vs. 

4  AH-64  NOE 

ZSU-23-4  52  95  4 

SA-11  25  50  0 

SA-13  25  40  0 

SA-14  13  23  0 


4  AH-64  200  ft. 


SA-14 

9 

16 

0 

1 

4.0 

8  AH-64  NOE 
SA-14 

11 

19 

0 

3 

8.5 

Blue  ADA  vs. 

4  Hind  NOE 

Chaparral  75 

Vulcan  58 

Stinger  30 


4  Hind  200  ft. 


Chaparral 

29 

46 

0 

3 

7.3 

Vulcan 

25 

40 

1 

2 

2160.0 

Stinger 

15 

30 

0 

3 

4.8 

(b)  Fixed-wing  aircraft.  Table  3  presents  the  results 
from  our  fixed-wing  aircraft  tests.  With  the  exception  of  the 
ZSU-23-4,  the  Blue  ADA  weapons  ate  all  more  lethal  than  their  Red 
counterparts.  Fixed-wing  analysis  has  previously  been  performed 
on  JESS  1.0.  However,  there  are  some  differences  between  the 
performance  of  ADA  in  JESS  1.0  and  in  JESS  1.1.  The  Patriot, 
Chaparral,  Stinger,  and  SA-14  are  performing  the  same  in  JESS  1.1 
as  in  JESS  1.0.  The  Hawk,  SA-11,  SA-13,  ZSU-23-4,  and  Vulcan  are 
killing  at  a  higher  rate  in  JESS  1.1  compared  to  JESS  1.0.  We 
attribute  these  higher  kill  rates  to  changes  made  to  the  JESS  1.1 
Pk  values  in  the  data  base.  (These  Pk  values  are  actually 
probability  of  hit  values,  but  they  are  called  Pk  values  in  the 
JESS  1.1  data  base.  We  will  refer  to  them  as  Pk  values  to  remain 
consistent  with  JESS  nomenclature) . 

b.  Comparing  the  benchmark  results  from  the  VIC  6.3  excursion  to 
the  JES^  1,1  AIA  results  for  fixed-wing  aircraft  revealed  the 
follow  >  : 

(1^  Hawk,  Chaparral,  Stinger,  SA-11,  ZSU-23-4,  and  Vulcan 
are  hitt'--  and  killing  fixed  wing  at  higher  rates  than  expected. 
Thesf  wea.  ..ns  are  two  to  three  times  more  lethal  in  JESS  1.1  than 
the  benchmark  results. 


100 

3 

6 

10.1 

83 

2 

4 

3240.0 

55 

0 

4 

8.5 

5  4000.0 

2  14.6 

4  9.5 

2  8.1 
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Table  3.  Fixed-wing  ADA  results  from  JESS  1.1 


Ave.  % 

Ave.  % 

Min.  no. 

Max .  no 

.  Ave. 

Weapon  vs.  tgt 

killed 

hit 

hit 

hit 

rds 

Red  ADA  vs. 

2  A-10  CAS  200  ft. 

SA-11 

40 

75 

0 

2 

15.1 

ZSU-23-4 

27 

50 

0 

1 

800 

SA-13 

15 

20 

0 

2 

2 

SA-14 

5 

10 

0 

1 

2 

2  A-10  BAI  200  ft. 

SA-11 

20 

85 

0 

2 

14.4 

2  A-10  CAS  50  ft. 

SA-13 

0 

15 

0 

1 

2 

2  F-16  BAI  200  ft. 

SA-11 

35 

40 

0 

2 

10.8 

ZSU-23-4 

15 

45 

0 

1 

800 

SA-13 

5 

10 

0 

2 

2 

SA-14 

0 

3 

0 

1 

1.8 

Blue  ADA  VS. 

2  Frog foot  CAS  200 

ft. 

Patriot 

70 

100 

2 

3 

4.5 

Hawk 

65 

100 

2 

3 

3.5 

Chaparral 

25 

55 

0 

2 

2.2 

Vulcan 

28 

40 

0 

1 

1080 

Stinger 

20 

30 

0 

2 

1.8 

2  Frog  foot  BAI  200 

ft. 

Hawk 

75 

100 

2 

3 

3.5 

4  Frog  foot  BAI  200 

ft. 

Hawk 

30 

63 

1 

4 

5.4 

10  Frog  foot  CAS  200 

ft. 

Chaparral 

5 

10 

0 

2 

2.2 

2  Frog  foot  CAS  500 

ft. 

Stinger 

25 

25 

0 

2 

2 

2  Frog  foot  CAS  1000 

ft. 

Hawk 

70 

95 

1 

3 

4.2 

2  Frog  foot  CAS  200 

ft. 

Hawk . 3 

55 

80 

0 

3 

4.7 

Sting . 3 

14 

19 

0 

1 

1.6 
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(2)  Patriot,  SA-13,  and  SA-14  are  killing  fixed  wing 
according  to  what  is  expected  from  the  benchmark  results.  Vulcan 
was  not  played  in  the  VIC  6.3  study,  but,  assuming  it  performs 
comparably  to  the  ZSU-23-4,  we  concluded  its  kills  were  also  too 
high. 


c.  The  average  number  of  shots  fired  per  engagement.  The 
last  column  in  tables  2  and  3  lists  the  average  number  of  shots 
fired  by  the  ADA  unit  per  engagement.  These  averages  are 
calculated  from  the  10  repetitions  for  each  ADA  test.  An 
engagement  by  the  ADA  unit  can  be  several  salvos,  depending  upon 
how  long  the  air  mission  is  within  range  of  the  ADA  unit.  Length 
of  time  the  air  mission  is  within  range  of  the  ADA  unit  depends 
upon  the  air  mission's  speed,  the  range  of  the  ADA  unit,  and  the 
location  of  the  air  mission's  path  across  the  ADA  unit's  lethal 
envelope. 


d.  The  average  percentage  of  air  missions  detected  by  the 
radar  unit. 

(1)  Army  aviation.  In  JESS  the  term  detection  as  it 
appears  on  the  ADA  spot  reports,  refers  to  detection  by  radar. 

An  enemy  target  can  be  detected  visually  and  shot  at  by  unit  X 
but  the  spot  report  will  read,  "Unit  X  shot  at  an  undetected 
enemy  mission."  Detection  by  radar  was  not  a  significant  factor 
in  our  helicopter  tests  since  the  ADA  weapons  we  tested  could 
rely  upon  visual  detection.  The  range  for  visual  detection  in 
JESS  is  eight  kilometers  (km) .  Effects  of  terrain  background  on 
weapons  are  not  modeled  in  JESS. 

(a)  If  an  Airborne  warning  and  Control  System  (AWACS) 
detected  an  enemy  mission  this  information  was  immediately  passed 
to  all  ADA  units.  An  AWACS  mission  always  detected  enemy 
helicopter  missions,  even  at  NOE  altitude.  This  can  be  changed 
via  a  data  base  value  called  mean. time. to. detect.  A  value  is 
entered  for  each  type  of  aircraft.  Setting  this  value  very  high 
eliminates  detection  by  AWACS. 

(b)  In  the  case  of  Stinger,  SA-14,  Vulcan,  and  ZSU-23-4, 
predetection  by  a  radar  unit  had  no  effect  on  the  attrition 
caused  by  the  ADA  unit.  Visual  detection  was  approximately  90 
percent.  If  the  aircraft  flew  within  eight  km  of  the  ADA  Stinger 
unit,  it  was  fired  upon  nine  out  of  ten  times. 

(2)  Fixed  wing.  If  an  AWACS  mission  was  on  station  all 
enemy  fixed-wing  aircraft  were  always  detected.  If  no  AWACS  were 
up,  the  SA-13  and  Chaparral  could  visually  detect  enemy  aircraft 
and  fire  upon  targets  that  were  flying  below  200  feet,  but  the 
Hawk,  Patriot,  and  SA-11  could  not  detect  nor  fire  upon  enemy 
aircraft  flying  below  200  feet  mean  sea  level  (MSL)  with  their 
radar,  nor  could  they  visually  acquire  the  target.  An  AWACS  had 
to  be  orbiting  for  the  Hawk,  Patriot,  and  SA-11  to  detect  and 
shoot  at  targets  flying  below  200  feet  MSL.  Even  when  the  Hawk 
unit  had  been  attacked  by  enemy  aircraft,  the  Hawk  would  not 
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shoot  unless  an  AWACS  had  detected  the  mission.  The  radars  for 
these  long-range  systems  have  minimum  detection  altitudes  in  the 
data  base  that  prevent  detection  below  0.1  km.  Detection  by 
AWACS  was  100  percent.  Detection  by  radar  without  an  AWACS  was 
about  90  percent  for  the  Chaparral  and  SA-13,  but  these  weapons 
could  also  rely  upon  visual  detection. 

e.  What  were  the  minimum  and  maximum  ranges  of  detection  for 
our  10  samples?  The  radar  detection  ranges  in  the  data  base  were 
checked  against  the  ranges  calculated  from  knowing  the  aircraft 
location  when  it  was  detected  and  the  location  of  the  detecting 
ADA  unit.  The  radar  values  were  always  within  the  range  limits 
set  by  the  data  base  values. 

(1)  If  the  aircraft  were  flying  at  500  feet  NSL,  the  Hawk 
radars  could  detect  them  no  farther  than  50  km  away.  The 
aircraft  were  usually  detected  at  the  farthest  edge  of  the  ADA 
radar's  detection  envelope.  The  Hawk  radar  usually  detected  the 
missions  when  they  were  40  to  50  km  away. 

(2)  Detection  of  the  enemy  unit  by  AWACS  occurred  as  soon 
as  the  air  mission  took  off  from  the  airbase.  The  data  base 
value  for  range  of  detection  by  AWACS  is  set  at  500  km. 

(3)  It  was  not  possible  to  identify  the  point  at  which 
visual  detection  occurred,  we  assumed  if  the  ADA  unit  fired,  it 
had  detected  the  mission.  We  could  track  location  at  the  time  of 
firing  by  the  ADA  unit  for  helicopter  missions  only.  These 
firing  range  distances  were  always  less  than  the  eight  km  visual 
range  of  the  weapon. 

f.  What  were  the  minimum  and  maximum  ranges  of  fire  for  our 
10  samples?  Table  4  lists  the  minimum  and  maximum  ranges  from 
which  the  ADA  weapons  were  noted  to  have  fired,  unfortunately, 
we  could  only  determine  firing  ranges  from  army  aviation  test 
results  as  their  spot  reports  are  the  only  ones  that  gave 
universal  transverse  raercator  (grid)  (UTM)  locations  of  the 
aircraft  when  it  was  engaged.  In  the  case  of  fixed-wing 
aircraft,  no  UTM  coordinates  were  given.  This  is  another 
difference  between  JESS  1.1  and  JESS  1.0.  In  the  older  version, 
the  UTM  coordinates  of  the  plane  were  given  when  it  was  shot. 

Table  4.  Recorded  firing  ranges  for  Red  and  Blue  ADA 


Max.  Min.  Average 

ADA  vs.  Mission _ (km) _ (km) _ (km) 


Stinger  vs.  Hind  at 

NOE 

6.5 

0.7 

3.0 

Chaparral  vs.  Hind 

at  NOE 

5.  7 

1.1 

3.4 

Vulcan  vs.  Hind  at 

NOE 

0.5 

0.1 

0.3 

ZSU-23-4  vs.  AH-64 

at  NOE 

4.0 

0.2 

1.8 

SA-13  vs.  AH-64  at 

NOE 

5.5 

0.1 

3.3 

SA-11  vs.  AH-64  at 

NOE 

30.0 

0.  5 

15.0 

SA-14  vs.  AH-64  at 

NOE 

5.5 

0.5 

2.7 
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(1)  From  the  ranges  we  determined  through  the  army 
aviation  reports,  we  found  that  all  but  the  ZSU-23-4  were  less 
than  or  equal  to  the  ranges  given  in  the  data  base  for  the 
maximum  effective  range  of  the  weapon.  The  ZSU-23-4  should  not 
fire  beyond  a  range  of  2.5  km  but  did  fire  at  a  helicopter 
mission  placed  at  a  four  km  standoff  range  from  the  ZSU  ADA  unit. 
We  have  no  explanation  for  this  occurrence. 

(2)  We  found  the  game  was  not  using  the  correct  range 
degrader  factors  for  very  low  altitude  bands.  The  range  of  the 
SA-11  and  the  SA-13  is  supposed  to  be  zero  if  the  aircraft  is 
flying  HOE.  However,  we  found  the  SA-13  could  shoot  AH-64  flying 
NOE  at  a  range  of  seven  km.  We  also  found  an  SA-11  could  shoot 
an  AH-64  flying  NOE  at  a  range  of  30  km.  Later  conversations 
with  personnel  at  JPL  confirmed  the  first  release  of  the  code  had 
a  problem  reading  the  range  degradation  factors.  We  also  found 
an  error  in  the  range  degrader  parameter  in  the  data  base.  The 
SA-14  and  SA-7  firing  against  aircraft  flying  NOE  should  be 
degraded  about  as  much  as  a  Stinger  firing  against  aircraft 
flying  NOE.  The  Red  weapons  are  degraded  to  0.08  of  their 
original  data  base  value  while  the  Blue  weapons  are  degraded  to 
0.8  of  their  original  value. 

g.  Other  needed  changes  are  as  follows:. 

(1)  The  fixed-wing  aircraft  sometimes  were  reported 
flying  at  negative  altitudes,  putting  them  below  ground  level. 
This  occurred  because  the  mission  orders  for  flying  altitude  are 
input  as  MSL  altitudes.  The  spot  detection  reports  give  the 
above-ground-level  altitudes  of  the  aircraft.  No  penalty  is 
assessed  in  the  game  for  flying  at  a  negative  altitude  because 
the  lowest  ADA  altitude  band  in  the  data  base  ranges  from 
negative  infinity  to  plus  490  feet.  Aircraft  may  be  at  an 
advantage  flying  below  ground,  because  the  range  degrader  bands 
work  on  above-ground  altitudes  and  the  ADA  weapon  range  may  be 
shortened  to  zero  if  the  aircraft  is  flying  at  or  below  NOE 
altitudes. 

(2)  We  also  discovered  an  unexpected  effect  on  helicopter 
attrition  caused  by  the  gamer's  method  of  inputting  the  mission 
order.  Taking  a  Red  helicopter  mission  as  an  example,  two  points 
were  specified  for  an  ingress  route,  starting  with  the  helicopter 
base  and  ending  with  the  target,  and  two  points  were  specified 
for  the  egress  route,  starting  with  the  target  location  and 
ending  with  the  helicopter  base  location.  The  mission  took  20 
minutes  to  arrive  at  the  target  and  only  three  Stingers  fired  at 
the  mission.  If  only  the  target  location  was  input  in  the 
ingress  menu  and  only  the  helicopter  base  location  was  input  in 
the  egress  menu,  the  time  to  arrive  on  station  was  83  minutes, 
and  the  Stinger  fired  nine  rounds  at  the  mission.  Furthermore, 
we  discovered  that  if  we  reversed  the  single  points  specified  in 
the  ingress  and  egress  menu  so  that  helicopter  base  location  was 
input  in  the  ingress  menu  and  target  location  was  input  in  the 


10 


egress  menu,  the  attrition  and  time  to  arrival  reverted  to  that 
obtained  from  specifying  the  two  points  for  ingress  and  two 
points  for  egress.  The  result  of  this  code  error  is  that  some 
missions  were  entirely  destroyed  due  to  the  gamers'  input 
procedure. 

9.  Sensitivity  analysis.  Tables  on  page  7  presents  results 
from  changing  the  mission  type,  the  mission  size,  the  mission 
altitude,  the  number  of  weapons  per  ADA  unit,  and  the  type  of 
aircraft.  Table  2,  page  6,  presents  results  from  changing  the 
helicopter  mission  size. 

a.  Effect  of  mission  type.  We  did  not  discern  a  significant 
difference  in  results  due  to  whether  the  fixed-wing  mission  was  a 
CAS  or  BAI  mission.  Table  3  shows  small  differences  in  the 
number  hit  when  Hawk  fires  at  a  Frog  foot  mission  flying  CAS  or 
BAI.  The  average  number  of  rounds  fired  per  engagement  and  the 
maximum  and  minimum  number  of  aircraft  hit  were  the  same  for  both 
CAS  and  BAI  missions  for  the  Hawk.  The  average  number  of 
aircraft  killed  in  the  SA-11  tests  for  CAS  vs.  BAI  is 
disturbingly  different.  We  attribute  this  difference  to  the 
randomness  in  the  damage  or  kill  determination  once  the  aircraft 
is  hit  and  not  to  the  fact  that  a  mission  was  flying  CAS  or  BAI. 
We  have  found  nothing  in  the  SDOL  code  or  the  data  base  to 
indicate  there  should  be  a  difference  in  attrition  depending  on 
whether  a  CAS  or  BAI  mission  is  ordered  by  the  gamer. 

b.  Effect  of  mission  size. 

(1)  Increasing  the  size  of  our  helicopter  mission  from 
four  ah-64  to  eight  AH-64  results  in  a  larger  number  of 
helicopters  killed  by  SA-14  (see  table  2) .  The  percent  killed  by 
SA-14  remained  approximately  the  same,  but  doubling  the  number  of 
helicopters  in  the  mission  means  the  number  of  helicopters  killed 
was  doubled  compared  to  the  number  killed  in  the  mission  with 
four  helicopters. 

(2)  Increasing  the  number  of  Frog  foot  aircraft  per 
mission  did  not  affect  the  number  of  aircraft  killed  or  hit  by  a 
Chaparral  unit  (see  table  3)  .  The  number  of  aircraft  hit  in  the 
ten-plane  mission  was  the  same  as  the  number  hit  in  the  two  plane 
mission.  Number  of  missiles  fired  was  also  the  same  for  the  two 
missions.  The  maximum  and  minimum  number  of  aircraft  hit  in  any 
one  sample  was  also  the  same  for  the  ten  aircraft  test  vs.  the 
two-aircraft  test.  The  code  is  structured  so  that  only  one  ADA 
weapon  type  per  unit  will  fire  at  a  mission.  After  firing,  the 
game  waits  a  minimum  time  to  fire  another  missile.  The  main 
advantage  to  having  several  weapon  systems  per  unit  is  that 
reload  times  will  not  constrain  the  attrition  results. 

(3)  The  total  number  of  fixed-wing  aircraft  hit  in  the 
10-sample  set  by  a  Hawk  unit  did  increase  from  20  to  25  aircraft 
when  the  number  of  aircraft  in  the  mission  was  increased  from  two 
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to  four  (see  table  3).  We  believe  this  is  a  significant  increase 
because  the  other  parameters,  such  as  average  number  of  rounds 
fired  per  engagement  and  maximum  number  of  aircraft  hit,  also 
increased  when  the  mission  size  was  increased. 

(4)  These  results  from  the  tests  on  mission  size 
indicate  that  helicopters  are  played  differently  from  fixed-wing 
aircraft  regarding  effect  of  mission  size  on  attrition  from  ADA 
units.  Helicopter  attrition  from  the  SA-14,  a  visual  weapon, 
doubled  trtien  the  mission  size  doubled.  In  the  case  of  fixed 
wing,  attrition  from  a  visual  weapon  did  not  increase  when  the 
mission  size  was  increased,  but  attrition  did  slightly  increase 
when  a  radar-guided  weapon  was  tested  against  a  larger  mission 
size . 


c.  Effect  of  altitude  on  helicopter  and  fixed-wing  attrition. 

(1)  Helicopters  flying  at  200  feet  have  a  better  chance 
of  surviving  than  those  flying  NOE  (see ‘table  2).  This  holds 
true  for  both  Red  and  Blue  helicopters.  We  believe  this  lower 
attrition  for  the  higher  altitudes  is  a  result  of  two  different 
effects  as  follows: 

(a)  The  speed  increase  given  to  helicopters  flying  at 
higher  than  NOE  altitudes.  Fewer  ADA  rounds  are  fired  at  the 
higher  flying  helicopters  because  they  are  flying  faster. 

(b)  The  ineffectiveness  of  the  range  degrader  bands  that 
should  decrease  ADA  fire  at  NOE  altitudes. 

(2)  Varying  the  altitude  within  the  bounds  we  set  for  the 
study  showed  some  effect  on  fixed-wing  attrition.  An  A-10 
mission  flying  at  5  0  feet  1<1SL  suffers  less  damage  from  an  SA-13 
than  a  mission  flying  at  200  feet  MSL.  This  is  due  to  the  range 
degradation  made  for  the  low  altitude.  The  SA-13  fires  only  one 
round  per  engagement  when  the  plane  is  flying  at  50  feet  MSL 
compared  to  firing  two  rounds  when  the  plane  is  flying  at  200 
feet  MSL.  Increasing  the  Frog  foot  mission's  altitude  from  200  to 
500  feet  MSL  had  no  effect  on  attrition  from  a  Stinger  unit. 

This  was  the  expected  outcome  since  the  data  base  Pk  remains  the 
same  from  0  to  500  feet  altitude  and  no  range  degradation  occurs 
for  this  altitude  increase.  Increasing  the  Frogfoot  mission's 
altitude  from  200  to  1000  feet  had  no  measurable  effect  on  the 
Hawk's  lethality.  We  believe  this  was  because  the  Hawk  was 
already  hitting  100  percent  of  its  targets  at  200  feet  so  there 
was  not  much  room  for  improvement  by  going  to  a  higher  altitude 
band  with  a  higher  Pk. 

d.  Effect  of  number  of  weapon  systems  per  ADA  unit. 

Changing  the  size  of  the  ADA  unit  to  a  smaller  nwber  of  Hawks 
and  Stingers  did  result  in  a  decrease  in  attrition  of  fixed-wing 
aircraft  but  not  to  the  extent  that  it  did  in  JESS  1.0.  Table  3 
shows  these  results  in  the  row  labeled  Hawk.  3  and  Stinger.  3.  In 
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these  tests  the  number  of  ADA  weapons  per  unit  was  reduced  to 
three  Hawks  and  three  Stingers.  Previous  tests  contain  six  Hawks 
or  nine  Stingers  per  ADA  unit.  In  JESS  1.0,  decreasing  the 
number  of  Hawks  to  one-half  resulted  in  half  the  attrition  to 
aircraft  compared  to  the  Hawk  ADA  unit  in  its  original 
configuration  of  six  Hawks.  In  JESS  1.1,  decreasing  the  number 
of  Hawks  by  half  resulted  in  only  a  20  percent  decrease  in 
attrition  of  aircraft.  Decreasing  the  number  of  Stinger  units  to 
one- third  the  original  number  resulted  in  a  33  percent  decrease 
in  attrition  rate  in  JESS  1.1.  in  JESS  1.0,  this  would  have 
caused  a  67  percent  decrease  in  attrition. 

(1)  The  guns  such  as  the  ZSU-23-4  and  the  Vulcan  are 
significantly  affected  by  reduction  of  number  of  guns  per  unit 
since  the  total  number  of  rounds  fired  per  salvo  is  dependent 
upon  the  number  of  guns  in  the  unit.  See  paragraph  9g.  for 
further  discussion  of  this  point. 

(2)  We  can  explain  the  decreased  attrition  for  the 
Stinger. 3  test  because  the  average  number  of  missiles  fired 
decreased  v^en  the  number  of  Stinger  missiles  per  unit  was 
reduced.  The  attrition  from  the  Hawk. 3  tests  decreased  because 
the  minimum  number  of  missions  hit  by  Hawk.  3  decreased  from  two 
to  zero.  Hawk. 3  tests  entirely  missed  hitting  one  air  mission 
and  had  one  less  total  destruction  of  an  air  mission  than  did  the 
tests  with  six  Hawk  systems. 

(3)  In  summary,  JESS  1.1  is  not  as  sensitive  to  reduction 
of  ADA  missile  systems  per  unit  as  was  JESS  1.0.  JESS  1.1  is 
still  very  sensitive  to  changes  in  the  number  of  guns  in  the  ADA 
unit . 


e.  Effect  of  type  of  aircraft. 

(1)  The  test  results  from  the  JESS  1.1  tests  show  lower 
attrition  for  the  F-16s  than  for  the  A-10S.  One  reason  is  the 
F~16  is  a  faster  aircraft  and  the  SA-11  ADA  does  not  fire  as  many 
rounds  at  the  F-16  as  it  does  the  A-1 0.  The  F-16  also  has  lower 
Pk  values  assigned  to  it  for  each  ADA  system  than  does  the  A-10. 

(2)  If  Pk  values  for  the  JESS  1.1  data  base  were 
developed  by  taking  into  consideration  the  aircraft's  speed,  then 
JESS  1.1  methodology  is  assessing  the  contribution  of  speed  on 
aircraft  vulnerability  twice.  The  variable  for  speed  controls 
how  long  the  aircraft  is  within  range  of  the  ADA  weapon  and  how 
many  salvos  will  be  fired.  Faster  flying  aircraft  take  fewer 
shots  and  accrue  less  attrition  as  seen  from  the  helicopter  tests 
in  this  report.  Helicopters  fly  faster  at  200  feet  than  at  NOE 
altitudes.  Table  2  shows  that  helicopters  flying  200  feet 
received  fewer  rounds  than  helicopters  flying  NOE. 
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£.  Effect  of  changing  the  range  on  attrition. 

(1)  Changing  the  range  of  the  ADA  weapon  may  affect 
attrition  if  the  range  change  affects  the  number  of  salvos  fired 
by  the  weapon.  Changing  the  number  of  salvos  fired  during  an 
engagement  is  similar  to  changing  the  number  of  tries  the  ADA 
unit  has  to  shoot  the  aircraft.  If  the  system  can  have  more  than 
one  try  at  hitting  the  aircraft,  the  probability  of  a  successful 
hit  will  increase  as  the  number  of  tries  increases.  Even  though 
a  weapon  system  may  have  a  low  Pk  (remember  Pk  means  probability 
of  hit  in  JESS) ,  the  overall  probability  of  at  least  one 
successful  hit  may  become  large  if  the  weapon  system  has  the 
opportunity  to  fire  multiple  salvos. 

(2)  All  the  above  discussion  simply  applies  what  we  know 
about  the  binominal  distribution  and  the  probability  of  a 
successful  hit  given  a  number  of  tries  and  given  the  probability 
of  a  success  with  just  one  try.  Figure  1  is  presented  to  help 
illustrate  this  point.  Each  line  shows  the  overall  probability 
of  success  given  a  single  shot  hit  probability  (Pk)  and  N  number 
of  tries.  From  this  graph,  one  sees  that  a  change  in  range  may 
or  may  not  have  a  linear  effect  upon  attrition.  If  the  Pk  is 
less  than  0.14,  the  effects  of  increasing  the  number  of  tries 
tends  to  be  linear.  Itie  higher  the  Pk,  the  less  linear  are  the 
effects  of  increasing  the  number  of  tries. 

(3)  The  methodology  in  JESS  1.1  looks  ahead  to  see  at 

what  time  an  aircraft  will  enter  a  radar  detection  circle  or  an 
ADA  weapon's  circle  given  its  input  flight  order.  These  event 
times  are  sorted  in  a  cue  and  the  process  "goes  to  sleep”  intil 
the  first  of  the  event  times  occurs.  The  game  checks  to  make 
sure  there  still  exists  a  capable  radar  or  ADA  weapon  to  detect 
or  shoot  at  the  aircraft.  If  so,  the  detection  process  or  the 
ADA  engagement  process  begins.  If  the  aircraft  is  within  range 
of  an  ADA  unit,  a  salvo  is  fired,  provided  the  weapon  is  loaded 
and  all  the  preconditions  for  fire  are  met.  The  ADA  unit  can 
refire  after  a  minimum  wait  of  15  seconds  if  the  air  mission  is 

still  within  range  and  all  preconditions  for  firing  are  met.  An 

air  mission  flying  390  knots  would  travel  approximately  three  km 
in  a  15- second  interval  so  an  ADA  weapon  with  a  range  over  three 
km  could  fire  a  second  salvo  after  a  15-second  wait  period. 

(4)  One  cannot  predict  that  for  every  three-km  range 

change  another  salvo  will  be  fired  because  the  wait  time  for  each 
ADA  weapon  is  also  dependent  upon  a  data  base  value  called  "wait 
time."  All  ADA  weapons  have  at  least  a  15-second  wait  time 
because  that  number  is  "hard  wired"  into  the  code  to  prevent  the 

attrition  routine  from  becoming  stuck  in  an  infinite  loop. 

(5)  Any  increase  in  firing  range  will  be  limited  by  the 
weapon  system's  detection  range.  A  Stinger  is  a  visual  detection 
weapon.  The  limit  on  the  detection  range  for  a  Stinger  is 
presently  set  to  eight  km.  increasing  the  stinger's  range  to  ten 
km  in  the  data  base  would  result  in  no  firing  range  increase. 
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Number  of  Tries 


Figure  1.  Probability  of  kill  vs*  number  of  tries 

g.  Effect  of  changing  the  number  of  rounds  per  burst  from 
guns.  For  the  ADA  guns,  such  as  Vulcan  or  the  ZSU-23-4,  there  is 
a  data  base  variable  called  number . rounds .per .burst .  The  actual 
number  of  rounds  fired  at  an  aircraft  is  a  function  of  this  data 
base  value.  The  methodology  for  guns  is  that  all  guns  shoot  at 
the  same  target  simultaneously.  Therefore,  the  number  of  rounds 
per  salvo  is  the  product  of  the  number  of  guns  times  the  data 
base  variable  number .rounds .per .bur st . 

(1)  Table  5  presents  results  from  Vulcan  and  ZSU-23-4  tests 
that  show  the  effect  of  changing  the  burst  rate  in  the  data  base 
on  the  average  attrition  to  helicopters.  The  last  two  columns  in 
table  5  show  the  actual  percent  hit  from  averaging  the  10 
replicate  runs  and  the  predicted  percent  hit  by  using  the 
equation  below  to  estimate  the  changes  in  attrition  from  changing 
the  burst  rate.  The  following  equation  is  used: 

1  -  (1  -  Pk)  **  n  where 

Pk  •  single  shot  probability  of  hit 
n  »  the  number  of  rounds  per  salvo 
(rounds  per  burst  *  number  guns) 
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Table  5.  Effect  of  number  rounds  per  burst  on  attrition 


No .  hit 


Burst 

rate 

Pk 

Rds./ 

salvo 

no. 

salvo 

out 
of  40 

%  hit 
actual 

%  hi 
pred  ic 

ZSU-23-4  vs. 

AH-64  (4)  NOE 

15 

0.0014 

60 

8 

3 

0.08 

0.08 

30 

0.0014 

120 

8 

5 

0.13 

0.16 

Vulcan  vs. 

Hind  (4)  NOE 

30 

0.0014 

120 

4 

5 

0.13 

0.16 

60 

0.0014 

240 

4 

8 

0.20 

0.29 

270 

0.0014 

1080 

4 

33 

0.83 

0.78 

100 

0.0007 

400 

4 

5 

0.13 

0.24 

200 

0.0007 

800 

4 

17 

0.43 

0.43 

(2)  Figure  2  was  generated  using  the  above  equation  to 
show  how  attrition  is  dependent  upon  number  of  rounds  per  burst 
and  the  single  shot  probability  of  hit  (Pk) .  Changing  the  number 
of  rounds  per  burst  by  decreasing  the  number  of  guns  in  a  unit 
will  have  a  significant  effect  on  attrition  by  the  guns. 

Normally  four  ZSU-23~4  will  be  located  in  a  unit.  Figure  2  helps 
predict  how  much  attrition  will  decrease  if  the  unit  loses  one  or 
more  of  its  guns. 
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(3)  The  predicted  values  agree  well  with  the  test  results 
except  for  the  100-round  burst  from  the  Vulcan.  This  test  result 
seems  too  low  given  other  test  results.  We  have  no  explanation 
for  this  one  unexpected  result. 

(4)  The  test  results  show  firing  four  salvos  vs.  eight 
salvos  did  not  have  a  significant  effect  upon  attrition.  The 
main  effect  on  attrition  came  from  changing  the  rounds  per  burst 
and  not  the  range  of  the  weapon  system  that  controls  the  number 
of  salvos  fired.  Figure  1  shows  that  the  slope  of  the  curve 
flattens  out  as  the  number  of  tries  for  a  successful  hit 
increases.  The  number  of  salvos  fired  by  the  Vulcan  and  ZSU-23-4 
was  already  high,  four  for  the  Vulcan  and  eight  for  the  ZSU,  so 
we  may  be  beyond  the  point  where  changing  the  number  of  salvos 
influences  the  outcome  significantly. 

h.  Effect  of  changing  the  Pk.  Table  5  and  figure  2  can  be 
used  to  examine  the  effect  of  changing  the  gun's  Pk  in  the  data 
base.  Changing  the  Pk  has  a  roughly  linear  effect  on  resulting 
attrition.  To  test  the  effect  of  reducing  the  Pk  on  missile 
performance,  the  Pk  of  the  Chaparral  was  reduced  from  0.5  to 
0.25,  and  the  resulting  attrition  was  half.  The  Stinger  Pk  was 
reduced  from  0.3  to  0.15,  and  the  resulting  attrition  was  half. 
All  other  parameters  were  held  constant  in  this  test.  If  the 
range  and  Pk  are  both  changed,  the  results  may  not  be  linear. 
Refer  to  paragraph  9f  where  effect  of  range  is  discussed. 

10.  Conclusions.  In  general,  we  cannot  conclude  that  ADA  is 
performing  according  to  any  acceptable  standards.  Comparing  the 
JESS  1.1  results  to  the  VIC  benchmark  published  in  Technical 
Memorandum  TRAC-F-TM-1388 ,  the  majority  of  the  weapons  we  tested 
were  far  too  lethal.  Changes  to  the  data  base  could  correct  some 
problems.  Other  problems  require  a  code  change.  Following  are 
specific  conclusions  classed  according  to  whether  the  conclusion 
relates  to  a  data  base  change,  a  code  change,  or  a  gamer  input 
option . 


a.  Data  base  changes. 

(1)  Comparable  Red  and  Blue  weapon  systems  such  as 
Stinger  and  SA-14  are  not  performing  comparably  in  attrition  of 
helicopters  or  fixed-wing  aircraft.  This  is  because  their  data 
base  Pk  values  are  very  different. 

(2)  The  Vulcan  and  the  ZSU-23-4  should  not  be  as  lethal 
as  they  are  against  helicopters  or  fixed  wing.  One  problem  is 
the  number  of  rounds  they  fire  per  burst  at  the  target  is  not  the 
data  base  value  for  rounds .per .bur st  but  rather,  the  product  of 
this  data  base  value  times  the  number  of  guns  in  the  ADA  unit. 

(3)  Stinger  and  Chaparral  attrition  rates  for  both  fixed 
wing  and  helicopters  are  too  high.  The  Chaparral  should  not  be 
firing  at  helicopters  flying  NOE. 
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(4)  SA-11  and  Hawk  attrition  rates  are  too  high.  Their 
Pks  need  to  be  lowered. 

(5)  Patriot,  SA-13,  and  SA~14  attrition  is  acceptable 
according  to  previous  benchmark  tests  with  VIC  6.3  excursion 
results  used  to  benchmark  JESS  1.0. 

(6)  Changing  the  weapon  system's  range  can  greatly  affect 
attrition  from  that  system  if  the  range  change  affects  the  number 
of  salvos  fired  per  engagement.  Systems  with  a  low  Pk  and  a  long 
range  can  still  have  a  high  attrition  rate  due  to  the  large 
number  of  tries  it  has  to  hit  the  target.  As  the  Pk  increases, 
changing  the  range  has  less  of  an  effect  on  resulting  attrition. 
Also,  if  the  number  of  tries  (corresponding  to  the  number  of 
salvos)  is  already  greater  than  four,  changing  the  range  will 
have  less  of  an  effect  on  attrition. 

(7)  Changing  the  number  of  rounds  per  burst  from  the  guns 
has  a  nonlinear  but  predictable  effect  on  attrition. 

(8)  Changing  the  weapon  system's  Pk  directly  affects 
attrition  from  the  weapon.  The  effect  is  dependent  upon  how  many 
tries  the  weapon  system  has  to  hit  the  target.  The  more  tries, 
the  less  will  be  the  impact  from  changing  the  Pk. 

b.  Code  changes. 

(1)  JESS  1.1  first  release  code  (referred  to  as  build  no. 
15  by  JPL)  is  producing  some  unintended  helicopter  attrition 
results.  According  to  the  documentation  in  the  SDDL  code  and  the 
data  base,  some  ADA  weapons,  such  as  the  SA-11,  SA-13,  Hawk,  and 
Chaparral,  should  not  be  shooting  at  helicopters  flying  NOE,  yet 
our  results  showed  these  weapon  systems  were  major  killers  of 
helicopters  flying  NOE. 

(2)  Helicopters  are  less  vulnerable  flying  at  200  feet 
above  ground  level  (AGL)  than  at  NOE  altitude.  This  is  an 
incorrect  representation  that  results  from  giving  the  helicopters 
a  large  speed  increase  when  they  are  flying  above  NOE  altitudes. 
This  speed  increase  is  controlled  by  the  code  and  is  not  a  data 
base  variable. 

(3)  The  number  of  guns  in  the  ADA  unit  significantly 
affects  'he  attrition  from  the  weapon  system  since  the  total 
rounds  fired  at  a  target  per  salvo  is  a  function  of  the  number  of 
guns  in  the  unit.  In  the  case  of  missiles,  the  number  fired  per 
salvo  is  not  dependent  upon  the  number  of  weapon  systems  in  the 
unit,  but  upon  a  data  base  value  unique  for  that  type  system. 
There  is  a  small  impact  on  attrition  when  the  number  of  missile 
systems  in  the  unit  is  reduced  if  readiness  of  the  weapon  system 
becomes  the  limiting  factor  in  determining  attrition  (i.e.,  the 
number  of  missiles  fired  is  constrained  due  to  reload  times). 
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(4)  Allowing  fixed-wing  aircraft  orders  to  use  MSL 
altitudes  but  requiring  the  program  to  select  degradation  factors 
based  upon  AGL  altitudes  causes  a  lot  of  confusion.  Gamers  think 
they  will  be  flying  at  200  feet  above  ground  when  they  input  the 
order,  but,  due  to  terrain  elevation,  the  mission  may  fly  at  a 
negative  altitude.  No  penalty  is  assessed  for  flying  at  a 
negative  altitude.  Some  reward  is  accrued  by  flying  at  a 
negative  altitude  because  certain  ADA  weapons  will  not  fire  if 
the  aircraft  is  flying  at  or  below  NOE  and  some  radars  cannot 
acquire  the  target  below  a  minimum  threshold  of  0.15  km. 
Helicopter  orders  are  always  input  as  AGL  altitude  so  this 
problem  does  not  occur  for  army  aviation  missions. 

c.  Gamer-controlled  input  variables. 

(1)  Size  of  the  mission  has  an  effect  on  both  helicopter 
and  fixed-wing  attrition  but  has  the  most  effect  on  helicopter 
attrition . 

(2)  Effect  on  attrition  from  altitude  of  the  aircraft  is 
mostly  attributable  to  effect  from  Pk  since  different  Pk  values 
are  selected  according  to  the  aircraft's  altitude,  in  the  case 
of  helicopters,  altitude  affects  attrition  through  selection  of 
Pk  by  altitude  band.  Also  altitude  selects  the  speed  of  the 
helicopter.  Helicopters  fly  faster  above  NOE  altitudes  and  have 
reduced  exposure  to  ADA  fire  due  to  their  increased  speed. 

11.  Recommendations.  Based  on  this  work,  we  recommend  that  JESS 
1.1  ADA  not  be  accepted  as  validated.  The  data  base  needs  to  be 
scrubbed,  and  the  obvious  inconsistencies  between  Red  and  Blue 
performance  for  comparable  systems  need  to  be  corrected. 

a.  The  Pk  values  for  the  Stinger,  Chaparral,  Hawk,  and  SA-11 
need  to  be  degraded  by  about  50  percent  for  altitude  bands  three 
and  four.  The  number  of  rounds  per  burst  for  the  Vulcan  and 
ZSU-23-4  needs  to  be  downgraded  to  60  rounds  per  burst.  The  Pk 
for  Vulcan  and  ZSU  needs  to  be  degraded  to  approximately  0.001. 

b.  We  recommend  code  changes  in  the  following  areas. 

(1)  The  range  degradation  factors  are  not  being  applied 
as  intended  due  to  a  code  problem. 

(2)  The  increase  in  speed  given  to  helicopters  flying 
above  NOE  altitudes  is  producing  some  unintended  results.  The 
code  needs  to  be  changed  to  prevent  helicopters  from  being  less 
vulnerable  at  200  feet  AGL  than  at  NOE  altitudes. 

(3)  The  definition  of  the  ADA  altitude  bands  in  the  data 
base  is  misleading.  The  data  base  values  are  overridden  by  the 
code  which  changes  the  boundaries  on  the  altitude  bands.  This  is 
very  confusing  for  people  supplying  Pk  data  based  upon  altitude 
of  the  aircraft.  Furthermore,  the  altitude  bands  for  the  range 
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of  the  aircraft.  Furthermore,  the  altitude  bands  for  the  range 
degrader  bands  are  defined  differently  from  those  for  the  ADA 
weapons,  i.e.,  instead  of  band  1  being  the  highest  band  it 
becomes  the  lowest  band,  and  the  values  for  the  bands  are  ceiling 
values  instead  of  floor  values.  Changing  the  code  so  there  is 
some  consistency  in  the  definition  of  these  bands  would  greatly 
aid  the  people  who  supply  data  for  these  variables. 

(4)  Fixed-wing  mission  orders  should  be  input  using  AGL 
altitudes  instead  of  MSL  altitudes.  Use  of  MSL  altitudes  allows 
the  aircraft  to  fly  below  ground  and  gives  the  aircraft  an 
unintended  protection  from  ADA  fire  since  the  ranges  of  the  ADA 
weapons  are  degraded  if  the  aircraft  are  flying  below  NOE 
altitudes. 
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